Eucalyptus: Integrating Natural Language Input with a Graphical User Interface
نویسنده
چکیده
This report describes Eucalyptus, a natural language (NL) interface that has been integrated with the graphical user interface of the KOALAS Test Planning Tool, a simulated Naval air combat command system. The multimodal, multimedia interface handles both imperative commands and database queries (either typed or spoken into a microphone) while still allowing full use of the original graphical interface. In this way the precision and consistency of direct manipulation is balanced and augmented by the descriptive power and reduced redundancy of NL. The two input media used together yield such powerful interaction techniques as deixis (simultaneous speech and pointing) and the ability to use mouse clicks and verbal referring expressions interchangeably. Finally, the system’s discourse handling capability allows abbreviated NL follow-ups (anaphora and ellipsis) to receive full interpretations based on the prior interaction context, whether verbal or graphical.
منابع مشابه
Two Multimodal Interfaces to Military Simulations
Eucalyptus and InterLACE are two projects integrating spoken natural language interaction with the already-developed graphical user interfaces of currently existing military simulation systems. We take this approach (rather than developing new, fully integrated systems from scratch) to facilitate the development of Navy demonstrations of NL technology applied to their own tools, as well as to e...
متن کاملA Dynamic Gesture Language and Graphical Feedback for Interaction in a 3D User Interface
In user interfaces of modern systems, users get the impression of directly interacting with application objects. In 3D based user interfaces, novel input devices, like hand and force input devices, are being introduced. They aim at providing natural ways of interaction. The use of a hand input device allows the recognition of static poses and dynamic gestures performed by a user’s hand. This pa...
متن کاملIntegrating Programming by Example and Natural Language Programming
We motivate the integration of programming by example and natural language programming by developing a system for specifying programs for simple text editing operations based on regular expressions. The programs are described with unconstrained natural language instructions, and providing one or more examples of input/output. We show that natural language allows the system to deduce the correct...
متن کاملCERC Technical Report Series Technical Memoranda CERC-TR-TM-92-001 KNOWLEDGE-DIRECTED GRAPHICAL AND NATURAL LANGUAGE INTERFACE WITH A KNOWLEDGE-BASED CONCURRENT ENGINEERING ENVIRONMENT
Effective on-line communication between cooperating users with the mediation of machines (Virtual Team concept) is a necessity in new-generation concurrent engineering design and manufacture systems. The communication means for solving this problem are natural language and graphics. Managing the graphical and natural language user interfaces is a task full of ill-structured problems for which t...
متن کامل